. DOCOHEHT BESOHB 



ED 146 22r 

&0THOR 
TITLE 



INSTITUTION 
SPONS AGENCY 
POB DATE 
GRANT 
NOTE 



EDfiS PRICE . 
DBSCBflPTORS 



TH 006 620 



IDENTIFIEi?S 



ABSTRACT*. 



Chapman^ Barren; And Others 

^SIGI: Field Test and Evaluation of a Computer-Based 
''Sys.tem of ' Interactive Guidance and Information'. 

Summary of Pinal Report* ^ 

Educational Testing Service, Prin^tdn, N.J. 
. National Science Foundation,^ ^ashii^gton, D.C.- 

Hay 77 

SED7i|-08971^A02 . 

19p.; For related documents, see TH 006 621 and 622 ; 
Best copy av-ailable' 

MF-$0.83 HC-$1. 67 Plus Postage. 

Career' iwareness; Career Choice; ♦Career Planning; 
College Students; Computer Oriented. Programs; Costs; 
Decisioi Baking; Decision Making Skills; Field 
Studied";. Formative Evaluation; *Guidance Programs;. 
Higher Education; Occupational Choice^ ♦Occupa'tional 
Guidance; *bn Line Systems; ♦Program Evaluation^; 
Summative Evaluation 

♦system of ifnteraptive Guidance and Information 



The compute r-'tjased System of I-iiteractive Guidance ajid 
information (SIGI) was field' tested and evaluated at Jive community 
colleges and one university. Developed by Educational Testing 
Service, SIGI assists students in tie process of informed ^ind 
rational career decision making. Interacting at a cathode-ray tube 
terminal with a qomputer, students ^exaoine .their , values, retrieve 
relevant inf6rmation about occupations, obtain predictive datk, 
^formulate plans, and learn decisioi) making strategies. Adopted as an. 
integral part.of'fhe career guidance and planning prograa at the 
field test colleges, SIGI ran smoothly, vas received enthusiast ically 
by students and counselors*, and proved effective in increasing 
students^ mastery of career decision making, competencies. This * " . 
,suttmary repott' briefly describes the Research ,d^sign and findings of 
the evaluation. (Author/B*)' ' • , - 



4c4c4c4c4c4c#4c ««♦«♦♦♦♦♦♦ 

♦ * Documents acquired by ERIC incl 

♦ material^ not available from other 

♦ to obtain the best copy available. 
*>/^producibility are^ cfften encounter 

♦ of the microfiche and' hardcppy repir 

♦ via- the ERIC Document Rep-rod,uction 

♦ responsible for t^e. quality of the * 
>♦ 'supplied by EDRS are the be^st tiiat 



ud^e many informal unpublished ♦ 
sources. ERIC makes every effort ♦ 
Nevertheless, items of marginal ♦ 
ed and .this affects the quality ♦ 
oductions ERIC makes available 
Service (EDBS) . EDBS is not ♦ 
original document'. Reproductions ♦ 
can be made from the original. ♦ 






* • 

1 




i 



il S OEPARTMgNTOF HfiALTH 

NATIONAL INSTirUTE OF 
60UCATI0N 

?^clcJ;^'*^''-'^ «6CE.VE0 FROM 
TM6 PERSON OR ORGANIZATION* ORIGIN- 
AriNG IT POrNTSOP Vr£W OR OPINIONS 
?PMrWcf? ""^^ NECESSARILY REPRE- 
SENT OPFir.AL NATIONAL UNSTlTUTE OF 
EDUCATION POSITION OR POLKV ' 



"PEF^MISSION TO REPRODUCE THIS 
MATERIAL HAS 6EEN GRANTED BV- 



: FIELD TEST AND EVALUATION 
■. OF A COMPUTER-BASED 
- SYSTEM OF INTERACTIVE • 
GUIDANCE AND INFORMATION 



TO THE EDUCATIONAL RESOURC€S 
i INFORMATION CENTER (ERIC) AND 
' USERS OF THE ERIC SYSTEM 




MAY 1977 




BEST mt^h^Mim 



WaiTcn Chapman 
''Nlartin R. Katz- 

Lila Norris 

Laura Pears 

In collaboratioh with: 
Emily Glossbrenner 
Amy Weber 




J 



Summary of .final report submitted to the National Science 
Foundation under Grant No. SED74-08971A02. Opinions, find- 
ings,,conclusions, or recommendations expressed in tHe report 
are those of the, authors and do no.t necessarily reflect-the views 
ofNfiF. ' ^ ' ' V • 

Educational Testing Service • Princeton, NJ. 




9 



Volume I of the >full report, SJGI: Field fest and ^Evaluation of a 
tlomikiter-B'^sed System of Interactive Guidance and Information , is 
available at .$20.00* per copy. Voliime 11, Appendices, is available 
at $15.00 per^ copy.. Send* prepayment or institutional purchase 
or<Jer dp: Order Service .Section, Educational' Testing Sewice, 
Princeton, '{Jew 'Jersey ' 08540. ^ . ' 

Copies are also available; under a separate fee^;sehedule, froiSi NationaL 
Technical Information Servic^, U. S, Department- of "^Commerce, Springfield, 
Virginia 221 61g^ - ^ • . . ' ^ - • ; \ 



• • ■ ^ . VOLUME I , ^; ; ' 

^ ' ■ ^ . * . . ,1 

CHAPTER • • ^ ' . * ^ " . 

I SIGIr Description and Rationale 

II Illustration of SIGI Interaction 

iU Selection of Occupations and Sources of Da^ for SIGI 

IV Evaluation Design * ^ " 

^ Y Findings at Illinois State University ' * ' . 

Vi Findings at Pasadena City College 

• VII Findings at. Santa Fe Community Colle^ 

VIII ' Findings at Eastfield College • 

IX Findings at Drf.ta' College ^ ' ^ - , 

< X Findings at Mercer County Community College \ 

. XI Summary and, D;iscussion of Findingfs aci;oss Colleges • ^ . . 

Xjl TUa Validity of- the SIGI Prediction System v 

Xlil Independent Studies ii)f the Effects of SIGI . , 

- XIV ^ Evaluative Highlights - " - . - ' 

;t. ... ' . ^ \ ^ ^hf^f'^, 

/V ' . > > - : ^ . • :-<^i:^r 

-v, , - VOLU>Ig II \^ : ; . 



APPENDIX 



A. Prediction System Manual 



B. Occupational Information in SIGI: A Handbook for 
, • Data Collection, Interpretation, Preparation,^ and 

Documentation * * , * ^ * 

C ' Planning System Manual . ' ' \. ^ . 

D SIGI Evaluation Instruments- \ 
E .SIGI lianager's Guide / / ' < 

F SIGI Hardware Guide 
G Counselor's* Handbook for SIGI , 



Copyrighf (C) 1977 by Educational Testing Service. 

' Alj, rights reserved. %r 



L 



I ■ ■ ■ • . * 

ABSTRACT 



A pomputer---based System of Interactive Guidance and Information (SIGI) 
w^s field-^tested and evaluated at five community colleges and one 
univ^ity. Developed by Educational testing Service, SIGI assists 
students in the process of informed and rational career decision-- . 
making. Inte;:acting at a cathode-ray tM^^ terminal with a computer, 
students examine their values, retr.ieve relevant information about 
occupations, obtain predictive data, formulate plans, and learn de-^* 
cision-making strategies. Adopted as an integral part of the career 
guidance and planning program at the field-test colleges, SIGI ran 
smoothly, was /received enthusiastically by students and counselors, 
and proved effective in increasing students' mastery of career de-; 
cision-making competencies. 
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SIGI: FIELD TEST AND EVALUATION OF. A COMPUTER-BJASED SYSTEM 
' OF INTERACTIVE GUIDANCE AND INF0R^LATI0^ '•^ 



. \^ Description of the System • - - 

The physic^al features of. SIGI include one of the PDP-11 series of com- ♦ 
puters manuf actur^by Digital Equipment Corporation and c^thode-tay tube 
terminals with typewriter-like keyboards by which students can respond to 
messages and information displayed on the sc^ean and can issue commanos to 
th^ computer* \ " ' ^ ' ^ ^ 

Tne content of SIGI consists of six interrelated subsystems called 
VALUES^ LOCATE, COMPARE, PREDICTION, PLANNING, and STRATEGY*' Taken in Che* 
order named, they comprise an organic system for career decision-making. ^ 
The system^ s|;arts with the student's own values — the rewards and satisfac- ^ 

tions he would like to realize through his occupation. In the Values system, 
he explores and examines his values, eventually assigning a numerical weight 
to each of ten occupational values to designate its importance to him. In 
Locate he commands the computer to assemble a list of occupations that meet 
or exceed his specifications on any set of fivfe of those values at a time. 
In Compare he asks pointed questions and gets information on all dimensions 
bf occupations of interesf. Prediction allows him' to calculate his chances 
of success, in the coursework that is preparatory for a particular occupation 
Planning shows him what_he should expect to do to qualify for entry into an 
•.occupation;, it maps pathways that l^ad to eacfi occupation. Strategy , . as the 
name implies, teaches decision rules based jointly on the rev^^rds offered by 
an occupation and the risks of failing to gat into it. ' 



T ^ 



The masciuline pronoun is' used in its generic sense for reasons of clarity 
and succinctness. It is, intended^ of course,, to refer to both females and 
males. ' , - ' - 
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J When the'stuWent has g^ne through these-subsystems ,in this order, he' 
becoffles an ^'-initiate*' and is turnei^loos^ to roam through- sfci at vill.^ . 
'The resources of SIGI'are now under his control: , He knows the patticular 

' ' ^ - ' • - . . ' r * ' . 

kind of informa^tion or assistance that resides in each subsystem .and he 

can go to the subsystem that meets his needs as he perceives them at the 

> * ' " ' . „ . V , . • . \ - * 

moment. ^ '' ' * / ' » ' 

Thus SIGI is tot;h a career guidance system and an information system* 

Its aim is to produce an autonomous* findividual capable of making informed 



and^ rational decisions^* The decision-maker '5. own examined values are the 
starting ^bint for rationality. They mediJbe between his self-concept ^ 
and the options avail^bj.e to him; they enable him to define and obtain in- 
' formation that is-relevant. ' ' * 

The emphasis is .hot merely on 'the content of^ the decisions but on the 
process' of decision-making. As the student progresses through SIG^, he 

. learns to move freely witHin the structure of the system. In gaining'. ^n- 

• V \ ^- ^ • ^ . ' ' ' . * ' y ' ' - 

trol of the system, he develops competencies and* masters, strategies for 

^» - • • .V 

, rational behavior, in the face of uncertainty — whi<:h may be th.e closest 'one 



can get to wisdom. . ' , * • ^. '^:> ' • ^ [\ ^ 

^ , • , *' Rese arch Design ' ' " f.'.. C':' 

r , »^ ^ a-- * ^.^ , - ' 

^Tb® field test ahd evaluation 'of SIGI took place -at six colleges repre- 

'Senting .graat , diversity in region, size, population, curriculum, and career 

guidance programs. The participating institutions were Delta College (Uni- 

versity Center, Michigan), Eastf^ield College '(Dallas , Texas), Illinois 
" ^ ♦ * ' - \ /.-^ - - ♦ ' ^ 

State Univergi-ty.' (Normal, Illinois) , ^Mercer County Community College (Tren- 

,Con, New Jersey), Pasadena^iity College (Pasadena, Calif orrria) ^ *and' 'Santa' 

Fe^ Community College ("fcainesyillfe, Florida). ^ ^ ' 



• . - /' 

The evaluation of SIGI was two-pronged: .'formative ^nd sumiHative, The 

purpose of the formative eval^tion was to improve the pperftion'of SIGI. 

The purpose of the silmmatiVe evaluation was to assess the impact of SIGI as 

a .career guidance sysWm, Sources of data included the students themselves, 

the counselors 'and other staff members, .and the records of student inter- 

action with the system. • • . '* 

Formative Evaluation \ ' >''v"?/!'. 

^ ^ • . ^ ^ - . ^. . ...... 

Jo improve, the operation of SIGI, we needed to know specifically 

(a) wh^t problei^s or gaps students found in the SIGI displays and content, 

(b) ' whether all/elements of the system were^^ed, (c) whether the "do-it- 



yourself" manuals coi;^ 'be used efficiently, (d) how valid the test-free 
Prediction system was, and (e) how SIGI could tie integrated with the total 



careec guidance program at each co\jLege» ^: V'^ . -* 

i ' - 

Questionnaires distributed to counselors and SIGI users included , 
items on the .clarity of dl^^tJlays and comprehensivene3S of information, 
SIGt .coordinators were asked to note any problems in the operation and use 
of the system.. Individual students* interactions with the system, recorded 
*by ^ the computer , were studied to determine whether use^ of the system was 
comprehensive and logicall;/ consistent. ' Small samples of* students at each 
college were intSirviewed to discover any additional problems or misconcep- 
tibns. \ ' \.- ' '.'/Tl'i. 'J-' 

The three manuals which had beeti developed to facilitate local control 
of the coijiput-er and the development of college-specific Prediction and Plan- 
ninV^systems were evaluated during the course of their use by the field-test 
colleges. The manuals are the SIGI Manager's Guide , the Prediction* System 
Manual , and the Pl-anning System Manual . 



The test-free Prediction system was evaluated not -only in terms of 
absolute, validities, but also in comparison with test-based predictions in 

• atudies done at two oS the colleges where test scores Vere available. Ob- 
servations, interviews, and counselor questionnaires 'were sources ^f' in- 
formation' about how SIGI was meshed into the total career guidance program 
at each institution and of opinions about how it could best be utilized. 
Summative Evaluation • . * . * * - ' : ' '^'-^J^pt^;:-'^ - 

The summative evaluation covered five major areas: (a) hardware r^- 
liability and /Cost, (b) students' reactions ^to 'SIGI, (c) the effects of SIGI 
ott students* career^ decision-making, (d) the impact of SIGI on counseling 
activities and guidance problems, and (e) 'summary data on students' iise of 

^SIGI. ' ' ' '~ . " . , !. * 

, *Co .determine tfhe reliability of'the SIGI hardware, we asked the field- 
test colleges to keep two sets of records of problems that affected the' 
operation of • SIGI during a three-month period. " For , one set, the computer 
-operators 'noted each hardware problem on' a ^peciaT form which was mailed to 
ETS Immediately.. The SIGI 'monitors Jcept a separate" log of m projilems' (in- • 
cluding-^iardware malfunctions) that iiiterfered with the normal operation of 
SIGI. This log. was collected at the end of -the test period. ' " -■"'^Bi^f:'^'*. • 

Questionnaires were administered to. random samples of SIGI user^Xex- " 
perimentais) and students who had not yet usee} it (coritrolsX to determine 
how SIGI affected the career ^aeciAon-mak-ing of experimental students and 
how -the experimentais differed from controls. In addition, between 10 and 
17 students who had used SIGI were interviewed at each college tq.gain'^ih- * 
sight into their career decision-making and to assess changes they experienced 
as a. result of using SIGI. - . / : . - 



Questionnaires were also distributed to counselors who worked with SIGI 
users or who worlced in C^he area o^* career counseling. The questionnaires 
were helpful in assessing ' the .nature of the-^ounselors ' interactions , with 
SIGI users, tlheir acceptance of SIGI, changes in counseling activities at- 
tributable to SIGI, and the impact of SIGI on problems in guidance, 

TJie^ .computer collected descriptive data oi^ the interaction of students 

. *^ • - ' 

with SIGI at each college. by automatically summing responses^ to -selected 

displays. These data were extremely useful* in showing the extent and patterns 

of use, " 



^ JIvaluative . Highlights 
Technical Aspects of tj;^^^ystem 

. Hardware ^liab/lity . The SIGI hardware is 
requires no special modification for SIGI. Its 



pletely independei?.t of SIGI. All six field- test 'colleges had slightly differ- 



ent hardware configurations, depending on the n 



all standard equipment .that 
reliability is^ ^therefore com-' 



mber of terminals and the 



. tasks imposed on the computer- in addition to running SIGI* After 'the 'instal- 
lation of SIGl and a short break-in period, al^ the sy^t^s'ran smobthly with 
no malfunctions traceable to Sl^GI. 

Cost. If the current cost of the hardw4re, software, and maintenance 
for a tasic f our-^e^ninal systemr is totaloQ over a .five-year period in which 
.each terminal is opera'ted 12 hours a da/ for 225 days per year (a rate of use 
easily sustained at the field-test sja:es), the cost per terminal h9ur would ^ 
be $1.73. Over the fiVer years, 13/500 students would be able Co use SIGI, 

'four Hours each; the cost .pf equ^ment would be $6.92 per-studentJ If tljie 

/ 

same assumptions are applied to/a 16-terTninal system, the cost^per terminal^ 



hour would b^ $0.90. 
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'Software/ The SIGI software, like the hardware, functioned sati^fac- 

torily. »The time-sharing capability was realized *befo^e the delivery of the 

hardware to the first user ^ college • The -response time at the system was 

excellent at the only college with as many as five, ierm^nals and in a test 

at ETS with si» terminals and acJditional peripheral? all in use at the same 

time.. The system was^£ree of bugs., and the- reprogramnied PredictioV system 

which pfermits -valid predictions without reliance on test scores, .opetated 

• » • » . - 

successfully at all colleges, - . * ' ^ 

Coursewate * ^ ♦ . ^ ' 

^per^tiOTx , The couraewate — the sequences oF displays that students 
actually .fc^ow — also proved entirely adequate, A large proportion of stu-'^ 
(ien^s expressed interest in the occupations retrieved for' them on the basis * 
of thei/T values. Almost all stud!ents were able 'to operate- the system without 
outside assistance and to' understand what they were doing. -^^V^v.. - 

Occupational information . /Both students and counselors, thought that 
the quality -of the occupational irifarmation was high. Three-quarter^ of . the. 



students indicated that it was better than information from other sources, 
;and only 1% thoaght it was worse/ The information cpvered all of'^^^fSJ -, P 
;the ar^as 'that students were interested in except for data about; local .'^ ' " 
salaries and bppcjrtuniti^ST- The SIGI data base included about 90% of the oc- * 
cugations 'that students named as b^ing of interest to them. Nevertheless, 
-plans continue for-ad^ng new occupatix)ns. ' / 

Acceptance by sKrudents . Students gave SIGI- high. grade?~86% graded it 
A ot B on liow interesting it was, 90% on how clear it ^s, and 87%. on Its 
overall "goodness.". The asperts of SIGI that concerned clarifying valUes, ' ' 
identifying occupations^ that Sit v^ues, and finding occupational; information 



received tjfie most A's and B's, and aspect^ that concerned prediction and plan-- 
ning received the fewest. Enthusiasm' for SIGI was high, and over 70% of the ^ 
SIGI users recommended SIGI to their friends. Over 60% of them wanted to use • 
the system again in .the future. . ' ' , 



" \ Reading level . Although some §tud^nts complained about the quantity of 

* * ^ ' * * fif$^ * ' * , - ' 

reading in SIGO:/ 92% df the, students found the* vocdt^lary and'style "Just 
: - ^ . ^ . . ' _ ■ ' • . . 

: ^ irigjit." .Only 1% found them "Too'dif f icult . " Only 'two' counselors out of 45' 

. . ^'^ ^ , ' ; ' - ^ . 

indicated tliiat the reading level was too hard for their students, one saying 

* *- 

o that dedf students experienced dif f icult^ widh some of the text and the other 
saying that fofetgn student's experienced sojae difficulty- 

freedom from bias . Over 98% of the students thought that SlGI was free 
• from sexual, racial, of other bias. .Forty-five out of 4^ counselors also^."^^ 
* thought th'kt^SIGI was bias-free. iSince' the students expr^ftsixig this 6pinion 
reflected the* sexual , racial, and -ethnic, composition of the college popula-*^ 
tion, it appears that nearly all members oflall groups, ^including women and 
' ^ minorities, believe that SIGI- is unbiased. \ * • / ' 



. . Problems and* revisions , "No single aspeat of SIGI stood out as a' problem 
^^-"•wj'— the SIGI users. Responses to questions dliout problems were scattered. ' ' . '"f,. 

d- 



Operating with a single-terminal* system ^ppeara to create problems in sched- 
iiling and in causing students to feel, rushed. Mor £he most part, students 
liked' Values , Locate,' and. Compare* more than ^Pred4ction^''^^lannihg, and 'Strat- 
% egy; but the systems they wpuld most like to use whexv tliey return were Pre- » 
diction and -Planning. 

Few revisions se^ necessary, in light of stujlettts* acceptance of SIGI. 
We intend, however, to expand the occupational' base. \ -We vould alsQ like to 
devise* a procedure that would allow a college to add VLocal oocupational 



information. Three technical chang^ are urtder way: increasing the number 

oE occupations that can be retrieved at o^e time in Locite, making .all 

' *■ . 

printouts op^tional, and simplifying the method of selecting*, occupations at * 

various points in die interaction • We would also lika to^taake a few re-^ 

finements in the courseware. ^ ^ ... 

Effectiveness of "Do="ItTYourself " Manuals • * , ' 

"*iMl three' manuals prepared by ETS to permit operation with a minimum* 

of technical assistance proved effective. The manuals are the SIGI^Manager 's 

Guide » the prediction System Manual , and 'the Planning System Manual .* The 

SIGI Manager's Guide was spff icieatly detailed to enable not just computer 

, operators but even ° technically unsophisticated staff members at some colleges 
. * ,• . ' 

to handle all' details of the day-to-day operation of SIGI. ' 

~^Th^ P^ddiction Sygtem >J:anual and Planning System 'Manual also proved 
sufficient. Following the detailed procedures in the mammals, each college 
was able Successfully to collec^: t;he data that result .in the course 'predic- 
tions in the Prediction system and to construct the displays that constitute 
their unique Planning systems. At all colleges the data collection and'" ♦ 

si i . • ' . * . \ ' 

preparation were done by^persons without special training in computer in- 
forxaation systjeins oj occupational information. T^e work was done by para- 
piof essionals and graduate students working with a regular- counselor ory hy" 
counselors themselves. Extensive consultatipn witH the SIGL staff was not 
required. All systems ran smocrthly and appeairdd to'-b'e of high quality. 

Development of the Predlqtion 'and Planning systems produced, a. dividejjd 

*• ^ .. ' , ■» 

for some colleges in the form of feedback • . Colleges were informed about j 

chapge$ in the distfibution , of .grades in: key courses and about instances 

when faculty 'selected grade factors tVigit turned oift to have little correla- * 



'tion with final grades. This information has stimulated reviews of policies- 
\ ' . ^-.^ ^ . . *^ 

concerning grades. The research reqtiired for the^ Planning pystem has albo 
been beneficial, resulting in ^tensive revision of the curriculum at '6ne 

* , - - % "y - • 

of the colleges and some changes at others • - ^ ' ^ 

• . ' C**/^ ' ^ > 

Another manual, the Counselor ^^itandbook for SlGfT ^as deemed i^seful 
as. a gaide< to follow-up sessions with st^Sllents who had used SIGI and also'<- 
as a resource foir career development courses • - ^. . • * " 

' '^'^i . ^ ' • . 

Development of 'Test-Free Prediction System 

Development of a Prediction system that would b6 independent ^ test 

scoi^es j^as successful^ "A tes t- free ^system was .necessary because onl}^ one i 

.field-'test college had a mandatory testing program. As a result of this. 

work, the SIGI predictions ''are based on/nonJ^M,.,,5iJC^dictor variables , such 

as previous academic p^erformance, the students' own s^elf-ratings on behaviors 

afsbciateowith .good grades in a 'particular course, and. the students' own 

^informed es^timates of their/final grades* The predictions may also be based 

on Eest scores when test scores are avkilabJLef Studies done at two of the 

colleges -wjiere test scores were available show that validities obtained with 

contest variables are generally as high as those obtained ^-Tith test scores 

and are OTme times higher^ * ' \ * \y\ J,?? 

Te'st^free predictions ofYer some benefits to the college. Prediction^ 
- ^ . ^ • ' - » • -.^ . . ^ ^ 

are'possible, at least for purposes of guidance, even .irt the absence t>i a 

testing program* Some courses that are hard to predict ^ith the usual 

measures of quantitative and verbal ability may be successfully predicted 

by nontest variables that' a<re especially selected to reflect the content of 

the course* Students become active rather than passive participants ±a the 



prediction process and for this reason may find it more acceptable than a \ 

testing pi:pgram. .Faculty .are induced tff examine and make explicit the bases 
« » 

.of their grades* Faculty also benefit from feedback showing them how w^ll 
• i ^ 

the grades they ^ctual^}^ awarded are correlated with the factors they speci- 
fied^as being important. \ ' - * r;-- 

'Usage of the System 

. Internal consistency . Summary data collected by the computer show th^t 
SIGI had a high degree of internal corisi^stency . The values that were mdst 
heavily weighted in the Values system were most^ frequently selected for ^^"^ ' 
trieval of occup^tions'^in the Locate system; occupations that were retrieved 
most often in Locate were generally amon§^ those most frequently selected^ for 
examination in the Compare and Strategy systems. ^ ^ 

Suf^ciency of individual subsystems . The subsystems 'did what they were 
designed to do. Each of* th,e ten values and each of the six intei^est 'fields 
were importcuit ,to some students, with .wide variation in the weights assigned 
"to each value. . Each (of the ten values w£Ls selected for use by 'some students 
ia the search for cpapatible occupation^ in the Locate system. Every QC-- 
cupation in SIGI was retrieved oby students at one or another of the colleges, 
indicating a .wjde- range in the values/specifications used by the students and 
corresponding differentiation in'* characteristics of occupations. As expected 
professional occupations, such asSeacher, Psychologist, Lawyer, Physician,* 
Dentis.t, and Speetsh Pathologist , '^re retrieved more frequently than non- 
professional ones, for the professional occupations tend to rate'-higher on 
the values given the greatest freight by these students. But students were 
selective, as indicated by the facts th|tt all occupations were retrieve(^ and 



^ . ■ -11- • 

that at one college .with a high proportion of 'older students, more nonprofes- 

W 

/ sioflal occupations were retrieved than at the other -colleges • Similarly, 

* every occupation in SIGI was selected for ejcamiriatioa in the Compare system, 

> and/every^ question was asked. Most of the programs for which 'predictions 

were available were called for in the* Prediction systems, and each of- the 
. ^ • . . • : ; ; - 

optional questions dhat students could ask about the preelection process was 

of interest ^to large numbers of students. Also, the da^a from the Strategy 

system show that students were influejaced byv^ infopnation aboOrt the desirabil- 

ity and »risks associated with occupations they had selected for study. 

Impact on Students - , . ' ' ^ ^ . 

^ Conclusions drawn from interviews * Interviewers looked for evidence ot 

Autonomy and rationality in the studetrcs' comments on their SIGI experience. 

They believe^d that the students had acquired a vocabulary for communicating 

about career choice. 8tudeuts tended 'to discuss occupations in terms of the 

' SIGI values. The dis\:overy that values provided -a basis for investigation 

as weLl' as a medium 'for communication gave them an awareness of structure in ^ ^ 

decision-making. ^They saw the logic ^'BiJ^^ghting their values ' and then usirfg 

> the ensuing knowledge* as a means for assessing occupations. Their behavior 
. , ' ^ ^ ' - . ' 

seemed purposive bol>h at the intellectual and behavioral levels; they seemed 

to have reasons for liking ox disliking'- an occupation, ; and they often took 
stepfe* toward a goal, such as changing programs, seeking outside help, ^or get- 
ting additional information. Also, the students 'had moved perceptibly for- 
ward in the process of making careier decisions. When they felt committed to 
a specific occupation, either the. one they had had irf mind before going on 
SIGI. or another, they felt, when they finished, that the commitment was well- 
grounded on information.' When they were .not committed to a specific occupa- 
. tiW at the beginning, progress .of ten consisted^of "an awareness th^t reasonable 



options existed and that they had learned an approach to assessing and either 
retaining or discarding options* . 

On the negative side, interviewers did not find that students hah com- 
pletely mastered'the SIGI algorithm for decision-making. Although, the sj:u- . 
dents were following the model and recognized that it wao^ rational, many of 
them would have had dif ficul^ty describing it to someone else. The inter- 
viewers did not pick up piuch evidence that students^ were able to generalize > 
from" the particular SIGI experience to other' kinds of decisions. 

Conclusfons drawn from questionnaires > Comparisons of responses to 
relevant questions on the cont*i:;ol and experimental questionnaires *§tiow num^r- 
ous significant differences between students who used SIGI and those who did 
not. More differences wer^ found at some coll'eges than at others. Yet there 



were great similarities in the way students responded to SIGI, despite the 



fact that the colleges were geographically dispersed and were also quite dis- 
similar in other respects • Pooling the ^data for al.l cplleges shows signifi-' 
cant differences (p^<'.01) between experimentals and controls in that SIGI 
users displayed a ^^ater knowledge^ of the rewards and satisfactions they want 
from an occupation, had more definite career plans, thought they could predict 
their grades better, kn^v^ better which program to enroll in for their occupa- 
tional goal, had greater confidence in their career decision-making 'ability, 
used the college reference library more f req,uai#|J.y , talked with guidance- 
counselors more often, used career-rSlated audiovisual. materials more fre- 
quently, rated themselves highe^^^as decision-makers and higher in .their know- 
ledge of occupations, had more accurate knowledge of the occupation they mi^ht 

* * 

enter, had seen a ^unselor in greater numb era within the previous tW6*weeks, 
and were more willing to interact with a computer.' Moreover", they would be 



^.05) less inclined to follow the advice of parents and friends, were less 

confused- by comflicting 'adv^^ce^, were less persuaded that knowledge of marriage 

plans was cruaial t£ career decisitDn-inaking, and had a clearer knowledge qf 
/ ' t * * * 

their values /and goals. * ^ . " 

Impact on Guidance Program. ' 

Acceptance. Responses th.eu^unselor questionnaire show a 'high degree 
of acfiept^Tice of compjuter-based guidance in general and SIQI 'in partituiar. 
C9unselors/ rejected by a wide margin the- notion tha't cbmputer-based guidance 
was '-a fad or >a ""threat, to them, and acdepted by a wide margin' the idea that 
such systems woul^ relieve them of ifoutine duties and would help students' 



.m^K^e care^er dec^-sio-ps. ^ 

Use Qf ■ thet systemy Only one counselor out of 57 'indicated that he or ^ 
she had' not actually referred students to SIGI. Al%the counselors who ob- 
served stu/ient response- to' SlSf -thought it was favorable. . 

" Impact on problem areas . vTounselq^ indicated that SIGI tiad had a 
favorable impact .of| one or anotl]^eKs,^f_^the are;gs identified as major^ or lienor 

problems. More counaelors thought that SIGI had had an impact on getting 
/ 4'.».'.' ' ' * . ' * ''y \' A' '*-t^M..,^-> 

> ' ' " . \ • ' "... 

students to read occupational, information and on keeping it up to date, which 

^ - . 1* ' . * * ' ' • - w - ■ 

w^e- the, most serious problem, arreas, thanjKhought it liad had an impact on 
less serioii's areas% ?ut each problem was designated by some counselor as » 



diminishing in seriousness becaus'e of the impact of SIG^. . ' 

Impact on counseling activities . .Counselors do not perceive SIGI as 

taking pver the burden of career counseling. Rajther, it Improves the quality 
V. • >t j * 

of counseling sessions. Students' airrive for thejir appointments with a better 
' • ' ^ / . ' ^ ' . • . 

backgr<)und, with bettet formuiatfj&d goals, with more accupations in mind, and 

with a more structured approach than do non-SIGt users.^ Counselors no4^ 

have to spend tioe^ providing background and* educating students in basic 



knowledge. Consequently, the session can be- devoted to matters of substance 
a^d the student can get more out of it, , ' 

How SIGI Fits in the Guidance Program 

^ Although the guidance programs at the six^lleges differed in many re- 
spects,* as did the colleges themselves, th^re were great similarities in the 
way students responded to SIGI and the impact that^ SIGI had on their career 
decision-^making. This fact suggests that SIGI is sufficiently flexible to 
be fitted into the counseling practices, of a college in various ways and still 
.remain effective, the "best" way of using SIGI, therefore, may' be the way 
that best suits the style of the college- * 

Nevertheless, studehts prefer a combination of SjS^I.and counselors for » 
help in aost .actiyities directed toward career choice. Counselors prefer a 
configuration in which they will- play a part in the students' choices, <There« 
fore it seems likely that a configuration which allows SIGI. and a counselor 
to supplement each other may be thevmost satisfying 'to all partie;s, if not / 
the most effective.' SJGI's role in this configuration is to provide an a]> 
gorithm for decision-making/ a V9cabulary for^ comjpunication, the basic steps 
of values clarification,' the retrieval of relevant information,- "and so on, ' ' 
The^ounselors/ role is to interpret, explain, anii supplement, as well as 
to help with personal problems. Using SIGI in conjunction with a course or 
seminar seems ^a logical arrangement because of the economies that can be 
obtained through* group counseling. The activities of the career information 
center should complement the SIGI--counselor combination so that SIGI, counse- 
lor, and center form a comprehensive' guidance program. SIGL should also be 
made 'available to students ad'libitura.^ ' • 



Independent Evaluations 

In addition to the research and evaluation cairried out by ETS, other 
studies of the effects of SIGL were conducted independently at a number of 
the user sites, most of them )by the cc^leges themselves. The res<ults of 
diese studies, undertaken because the colleges wanted to find out in their 



own terms how well SIGI was meeting their needs, reiterate the main theme.s 

' . ' N; 

emerging from this report: ^satisfaction with SIGI b,y students and coun- 
selors; evidetice of effectiveness, however defined, ^n meeting stated 
objectives'; side effects in enhancing counselors' contributions to ^ 
'students' career decision-makin'g. The evjldeh^e appears not only in ques- 
tionnaires and tests; it* comes also ff9m such unol^rusive measures as ^ong 
waiting lists to use SIGI, the high proportion of users who have be,en re- 
ferred by their friends and who spread -the^-word^ to other friends, thjfe de- 
veXbpment of courses and various activities career planning Centers to 
fociis on SXGI and integrate it into the total career guidance program, the . 
enthusiastic reception of visitors, and eagerness to see SIGI used as widely 
as possible. , * * . , r\ 

,,The fact that the p;:esence of SI^I has prompted independent evaluation 

and research studies on caiapuses is itself a bonus. It. is useful for col- 

' ' ^ ' • ' ' " - ' - ' ' ^ ■ ^'/ r'' ' - 

leges to define their objectives in career guidance, to plan programs, to 

c^.. \ ' ' ' ' i ' " ' * 

cpll^^ct and analyze evidence of effectiveness. Thus, SIGI may serve as a • 

catalyst in the process of rational and informed,. decision-making by the 'col- 

leges themselves as they strive to meet th'e career ,devel9pment needs of 

students. ^ . . " ' - 



